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Introduction 


With  the  emergence  of  Unmanned  Aerial  Vehicles  (UAVs)  and  the  video  payloads  now  available 
onboard,  real-time  surveillance  of  dynamic  situations  is  now  possible.  Air  Force  commanders 
are  often  required  to  make  critical  decisions  during  these  dynamic  situations  based  on  limited 
information.  In  order  to  increase  the  situational  awareness  of  commanders,  we  have  developed 
a  prototype  UAV  Situational  Awareness  Tool.  This  tool  will  connect  the  commander  in  a 
command  center  to  an  operational  UAV  in  the  field. 

The  prototype  tool  we  have  developed  is  based  on  a  Geographic  Information  System  (GIS) 
displaying  a  map  of  the  situation  area  along  with  a  video  feed  from  the  UAV.  By  displaying  the 
location  of  the  UAV  on  the  GIS  map  along  with  the  video  feed  from  the  UAV,  commanders  will 
have  a  bird’s  eye  view  of  dynamic  situations.  Using  this  tool  in  a  command  center  during 
exercises  will  increase  the  situational  awareness  of  Air  Force  Commanders. 

In  his  concept  paper  [1],  Harper  describes  a  dynamic  situation  that  includes  two  separate 
emergency  events  occurring  on  the  Air  Force  Academy.  As  the  base  commander  responses  to 
these  events,  it  is  imperative  to  have  accurate,  real-time  data  about  the  situations.  By 
combining  existing  technologies  in  GIS  systems,  UAV  video  feeds,  and  networking  technology, 
our  prototype  tool  provides  the  commander  a  real-time  video  feed  from  a  single  UAV.  This 
video  feed  provides  the  commander  real-time  visual  information  that  will  greatly  increase  his/her 
situational  awareness.  This  prototype  tool  addresses  Harper’s  concept  paper  [1]  and  provides 
an  initial  capability  for  response  to  these  types  of  situations. 

Research  Objectives 

Display  a  UAV  icon  in  a  GIS  Application 

The  first  objective  is  to  build  a  GIS  application  that  interfaces  with  the  UAV’s  onboard  autopilot. 
The  GIS  application  needs  the  current  position  of  the  UAV  in  order  to  show  the  correct  location 
on  the  map.  This  information  is  gleaned  from  the  telemetry  information  being  received  from  the 
UAV. 

Display  the  Video  Feed  from  a  UAV 

The  next  objective  is  to  capture  the  video  stream  being  sent  from  the  UAV  and  embed  it  in  the 
GIS  application. 

Provide  the  UAVSAT  Tool  to  Commanders  in  a  Command  Center 

The  final  objective  is  to  connect  the  commander  in  the  command  center  to  the  operational  UAV 
flying  in  the  field.  We  developed  a  client/server  architecture  to  support  this  objective. 

The  UAV  Situational  Awareness  Tool 

Figure  1  shows  a  screen  capture  from  the  prototype  UAV  Situational  Awareness  Tool 
(UAVSAT).  The  tool  is  based  on  existing  GIS  technology  [2]  which  allows  the  user  to  overlay 
any  number  of  information  layers  onto  an  existing  map  object.  GIS  systems  are  currently  being 
used  by  security  forces  to  display  maps,  secure  cordons,  response  vehicles,  etc,  during 
emergency  situations.  Figure  1  shows  the  video  stream  from  the  UAV  on  the  left  of  the  screen 
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and  the  location  of  the  UAV  on  the  right  side  of  the  screen.  The  video  window  is  based  on 
Microsoft’s  Windows  Media  player  ™.  This  window  is  a  dockable  window  that  can  be  detached 
from  the  main  UAVSAT  window.  This  allows  the  user  to  display  the  video  on  one  screen  and 
the  GIS  map  image  on  another  if  multiple  screens  are  available. 
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Figure  1  Screen  Capture  from  UAVSAT 
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The  UAVSAT  tool  includes  a  UAV  icon  in  the  GIS  display,  which  moves  around  the  map  and 
shows  the  current  location  of  the  UAV  during  flight.  The  latitude  and  longitude  of  the  UAV, 
taken  from  the  UAV’s  telemetry  data,  are  used  to  geolocate  the  UAV  on  the  map  display.  In 
Figure  1,  this  icon  is  shown  currently  southeast  of  the  Chapel. 

Figure  2  shows  the  overall  system  architecture  we  used  to  provide  both  the  video  and  the 
telemetry  data  to  the  UAVSAT  tool. 
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Figure  2  Overall  System  Architecture 

The  telemetry  from  the  UAV  is  sent  from  the  onboard  autopilot  to  the  Ground  Control  Station 
(GCS)  via  a  900  MHz  channel.  This  telemetry  includes  latitude,  longitude,  altitude,  pitch,  roll, 
yaw,  etc.,  as  described  in  the  Piccolo  communications  documentation  [3],  The  UAV  telemetry  is 
sent  to  the  server  and  converted  into  a  form  that  is  sent  to  all  the  clients  via  a  program 
developed  for  this  project.  On  a  separate  stream,  the  video  images  are  sent  from  the  onboard 
camera  to  the  receiver  on  the  ground  via  a  2.4  GHz  channel.  These  video  images  are  sent  to 
the  server  and  converted  to  a  format  that  is  capable  of  being  streamed  to  the  clients.  This 
process  is  done  by  the  Microsoft  Windows  Media  Server.  This  process  can  take  from  six  to  nine 
seconds  to  perform,  which  results  in  a  delay  of  six  to  nine  seconds  from  the  time  the  video  was 
shot  onboard  the  UAV  to  the  time  it  can  be  displayed  in  the  command  center.  The  clients 
connect  to  the  server  via  an  Internet  Protocol  (IP)  connection  and  receive  telemetry  and  video 
information  over  IP. 

Integrating  video  into  a  GIS  application  is  not  a  new  technology.  However,  the  authors  have  not 
found  a  system  that  integrates  real-time  video  with  telemetry  data  into  a  GIS  application  using 
an  Internet  connection.  The  Red  Hen  Systems  Corporation  [4]  provides  a  video  capture  and 
playback  facility  for  ESRI  products  such  as  ArcMap  and  ArcGIS,  but  this  is  not  a  real-time  video 
feed.  Their  GeoVideo  application  provides  a  means  of  capturing  geo-referenced  DVD  video 
and  the  ability  to  replay  the  video  in  ArcGIS  applications.  Geo-referenced  still  images  can  be 
captured  during  playback  of  the  DVD  video  [4].  The  GeoVideo  application  was  used  by 
students  from  the  University  of  Nebraska  to  build  gNet,  an  interactive  map  of  the  Lewis  and 
Clark  explorations.  [5]  The  gNet  systems  is  not  a  real-time  video  system.  Similarly,  the  Iwane 
Video  GIS  system  [6]  allows  the  user  to  view  digitized  videos  of  selected  locations,  but  it  is  not  a 
real-time  video  system. 


5 


UAVs  at  the  Air  Force  Academy 

The  UAVs  used  for  this  research  are  based  on  the  Piccolo  autopilot  system  as  shown  in  Figure 
3.  This  autopilot  system  includes  a  Software  Developer’s  Kit  (SDK)  that  can  be  used  to  develop 
specialized  UAV  applications  based  on  the  C++  programming  language. 


Figure  3  Piccolo  Autopilot 

The  Piccolo  autopilot  is  used  in  such  UAVs  as  the  Silver  Fox,  Pointer,  and  Aerosonde.  Due  to 
operational  and  financial  limitations,  we  have  based  our  research  on  a  UAV  built  from  the  Alpha 
60™  remote  controlled  model  airplane  [7].  Figure  4  shows  a  UAV  based  on  the  Alpha  60, 
which  has  been  configured  to  fly  with  a  Piccolo  autopilot.  The  same  Ground  Control  Station 
(GCS)  that  is  used  for  the  Silver  Fox  UAV  is  used  to  control  the  Alpha  60-based  UAV.  Due  to 
altitude  limitations  at  the  Academy,  we  also  add  a  larger  engine  to  the  Alpha  60. 


A  system  for  video  capture  was  developed  under  a  previous  grant  to  Lt  Col  Sward  [8],  Flight 
tests  with  this  inexpensive  camera  payload  have  shown  that  a  camera  with  better  resolution  and 
range  was  needed.  It  is  especially  crucial  to  have  a  higher  resolution  camera  for  video  capture 
missions.  It  is  also  crucial  to  have  reasonable  range  capability  for  the  camera  system.  During 
the  earlier  video  capture  missions,  the  signal  between  the  camera  transmitter  and  receiver  was 


6 


often  lost.  This  resulted  in  distracting  flashes  of  static  and  black  screens  in  the  video  capture 
image.  These  limitations  will  be  magnified  during  real-time  crisis  situations  when  the 
commander  needs  good  quality  video  signals. 


Figure  5  Color  CCD  Camera 

To  solve  the  resolution  problem  with  our  existing  video  capture  system,  we  purchased  the 
camera  shown  in  Figure  5.  This  camera  is  a  color  Charged  Couple  Device  (CCD)  camera  with 
450  TV  lines  of  resolution.  This  provided  better  resolution  for  video  capture  missions. 


Figure  6-Video  Transmission  System 


To  solve  the  range  problem  with  our  existing  video  capture  system,  we  purchased  the  video 
transmission  system  shown  in  Figure  6.  This  system  provided  a  greater  range  for  our  video 
capture  system. 

Additional  Results 

The  15  cadets  enrolled  in  the  computer  science  course,  CS-453,  Software  Engineering,  were 
divided  into  three  groups,  each  composed  of  a  team  leader,  a  design  engineer,  and  two 
programmers.  The  teams  consisted  of  a  Field  of  View  Team,  a  Streaming  Media  Delay  Team, 
and  an  Automatic  Orbit  Establishment  Team.  An  integration  officer  was  responsible  for 
integrating  the  work  of  the  teams.  Overall,  a  program  manager  and  an  assistant  managed  the 
work. 

At  specified  intervals  during  the  course,  the  groups  were  required  to  produce  reports  of  progress 
on  assigned  work,  including  achievements  and  challenges.  In  fulfilling  the  multi-faceted 
requirements,  the  cadets  participated  in  developing  software  as  a  part  of  a  large  development 
team,  combining  project  management,  programming,  integration,  and  system  deployment  skills. 
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Work  Remaining 


Because  of  the  processing  time  required  for  Microsoft  Windows  Media  Server  to  prepare  the 
video  for  streaming,  there  is  a  time  delay  introduced  between  when  the  video  was  shot  onboard 
the  UAV  and  when  it  is  presented  to  the  client  applications  in  the  command  center.  This  delay 
can  be  from  six  to  nine  seconds  in  length  and  is  not  a  constant  length.  Because  of  this  delay, 
there  is  a  synchronization  problem  between  when  the  video  was  shot  and  where  the  UAV  is 
shown  on  the  GIS  map  display.  Since  there  is  little  delay  in  the  transmission  of  the  UAV’s 
location,  the  difference  between  where  the  UAV  is  shown  on  the  map  and  the  location  that  the 
video  is  currently  showing  can  be  sizable.  Future  research  should  work  on  minimizing  this  delay 
and  this  synchronization  problem  between  the  location  of  the  UAV  on  the  map  and  the  video 
stream. 

Currently,  the  distance  between  the  video  receiver  and  the  server  computer  is  not  great.  In 
order  to  deploy  this  system  to  the  field,  there  must  be  some  way  to  increase  the  distance 
between  where  the  video  signal  is  being  captured  by  the  receiver  and  where  the  server  is  that  is 
streaming  the  video  signal  to  the  clients.  Future  research  should  work  on  extending  the  distant 
between  the  receiver  and  the  server. 

Project  Summary 

Overall,  the  project  was  a  success  in  that  we  developed  a  prototype  system  that  demonstrates  it 
is  possible  to  connect  the  commander  in  a  command  center  with  the  video  from  and  the  location 
of  an  airborne  UAV.  The  UAVSAT  tool  incorporates  UAVs  as  another  layer  of  information 
available  to  the  commander  in  his/her  GIS  toolkit.  This  combination  of  the  video  and  the 
location  of  the  UAV  is  a  powerful  asset  for  the  commander  during  a  crisis  situation. 
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